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In this paper we study classes of graphs with three types of edges that capture the modified 
independence structure of a directed acyclic graph (DAG) after marginalisation over unobserved 
variables and conditioning on selection variables using the m-separation criterion. These include 
MC, summary, and ancestral graphs. As a modification of MC graphs we define the class of rib- 
bonless graphs (RGs) that permits the use of the m-separation criterion. RGs contain summary 
and ancestral graphs as subclasses, and each RG can be generated by a DAG after marginalisa- 
tion and conditioning. We derive simple algorithms to generate RGs, from given DAGs or RGs, 
and also to generate summary and ancestral graphs in a simple way by further extension of the 
RG-generating algorithm. This enables us to develop a parallel theory on these three classes and 
to study the relationships between them as well as the use of each class. 
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1. Introduction 

Introduction and motivation. In graphical Markov models graphs have been used to 
represent conditional independence statements of sets of random variables. Nodes of the 
graph correspond to random variables and edges typically capture dependencies. Different 
classes of graphs with different interpretation of independencies have been defined and 
studied in the literature. 

One of the most important classes of graphs in graphical models is the class of directed 
acyclic graphs (DAGs) [4; 6]. Their corresponding Markov models, often knowrn under 
the name of Bayesian networks [8], have direct applications to a wide range of areas 
including econometrics, social sciences, and artificial intelligence. When, however, some 
variables are unobserved, that is also called latent or hidden, one can in general no longer 
capture the implied independence model among observed variables by a DAG. In this 
sense the DAG models are not stable under marginalisation. A similar problem occurs 
because DAG models are not stable under conditioning [16; 9; 2]. 

This makes it necessary to identify and study a class of graphs that includes DAGs and 
is stable under marginalisation and conditioning in the sense that it is able to express the 
induced independence model after marginalisation and conditioning through an object 
of the same class. The methods that have been used to solve this problem employ three 
different types of edges instead of a single type. 

Three known classes of graphs have previously been suggested for this purpose in 
the literature. We specifically call these stable mixed graphs (under marginalisation and 
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conditioning) and they include MC graphs (MCGs) [5], summary graphs (SGs) [16; 13], 
and ancestral graphs (AGs) [10]. 

MCGs do not use the same interpretation of independencies, called the m-separation 
criterion, as the other types of stable mixed graphs. In this paper we use similar methods 
as in [5] to derive a modification of the class of MCGs to use m-separation, which we 
call ribbonlcss graphs (RGs). The class of RGs is exactly the class with three types of 
edges that is generated after marginalisation over and conditioning on the node sets of a 
DAG. More importantly, we extend the RG-generating algorithm to generate summary 
and ancestral graphs in a theoretically neat way. These algorithms are computationally 
polynomial, even though we shall not go through their computational complexity in this 
paper. Defining these algorithms leads to establishing a parallel theory for the different 
classes, and studying the similarities, differences, and relationships among them. 

Structure of the paper. In the next section we define some basic concepts of graph 
theory and independence models needed in this paper. 

In Section 3, we define the class of RGs, give some basic graph-theoretical definitions 
for these, and define the m-separation criterion for interpretation of the independence 
structure on them. 

In Section 4, we formally define marginalisation and conditioning for independence 
models in such a way that it conforms with marginalisation and conditioning for proba- 
bility distributions. We also formally define stable classes of graphs. 

Each of the next three sections of this paper deals with one type of stable mixed 
graphs. We discuss RGs in Section 5, SGs in Section 6, and AGs in Section 7. In each 
section we introduce a straightforward algorithm to generate the stable mixed graph from 
DAGs or from graphs of the same type. For each type of stable mixed graph, we prove 
that the graphs and algorithms are well-defined in the sense that instead of marginalising 
over or conditioning on a set of nodes, by splitting the marginalisation or conditioning 
set into two subsets, one can marginalise over or condition on the first subset first, and 
then marginalise over or condition on the second subset and obtain the same graph. 
We also prove that the generated graphs induce the modified independence model after 
marginalisation and conditioning, meaning that the generated classes are stable under 
marginalisation and conditioning. 

In Section 8, we scrutinise the relationships between the three types of stable mixed 
graphs. In Section 9 we provide a discussion on the use of the different classes of stable 
mixed graphs. 

In the Appendix we provide the proof of lemmas, propositions, and theorems given in 
the previous sections. 

2. Basic definitions and concepts 

Independence models and graphs. An independence model J over a set F is a set 

of triples {X,Y\Z) (called independence statements)^ where X, Y, and Z are disjoint 
subsets of V and Z can be empty, and {0,Y \Z) and (X, | Z) are always included in J . 
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The independence statement {X, Y \ Z) is interpreted as "X is independent of Y given 
Z" . Notice that independence models contain probabihstic independence models as a 
special case. For further discussion on independence models, see [12]. 

A graph G is a triple consisting of a node set or vertex set V, an edge set E, and 
a relation that with each edge associates two nodes (not necessarily distinct), called its 
endpoints. When nodes i and j arc the cndpoints of an edge, these are adjacent and we 
write i j. We say the edge is between its two cndpoints. We usually refer to a graph as 
an ordered pair G = {V,E). Graphs Gi = (Vi,£'i) and G2 = (V2,i?2) are called equal if 
{Vi,Ei) = {V2,E2). In this case we write Gi = G2. 

Notice that the graphs that we use in this paper (and in general in the context of 
graphical models) are so-called labeled graphs, i.e. every node is considered a different 
object. Hence, for example, graph i j k is not equal to j i k. 

We use the notation j'^ for an independence model defined over the node set of G. 
Among the independence models over the node set F of a graph G, those that are of 
interest to us conform with G, meaning that « ~ j in G implies (i, j | G) ^ J7 for any 
G C F \ {«, j}. Henceforth, we assume that independence models j'^ conform with G, 
unless otherwise stated. Notice that henceforth we use the notation i instead of {1} for a 
subset consisting of a single element i in an independence statement. 

Basic graph theoretical definitions. Here we introduce some basic graph theoretical 
definitions. A loop is an edge with the same cndpoints. Multiple edges are edges with the 
same pair of endpoints. A simple graph has neither loops nor multiple edges. 

If a graph assigns an ordered pair of nodes to each edge then the graph is a directed 
graph. We say that the edge is from the first node of the ordered pair to the second one. 
We use an arrow, j — t-z, to draw an edge in a directed graph. We also call node j a 
parent of i, node i a child of j and we use the notation pa(i) for the set of all parents of 
i in the graph. 

A walk is a list (vq, ei, ui, . . . , e^, Vk) of nodes and edges such that for 1 < i < fc, the 
edge Ci has endpoints and Vi. A path is a walk with no repeated node or edge. A 
cycle is a walk with no repeated node or edge except wq — Vk- If the graph is simple 
then a path or a cycle can be determined uniquely by an ordered sequence of node sets. 
Throughout this paper, however, we use node sequences for describing paths and cycles 
even in graphs with multiple edges, but we suppose that the edges of the path are all 
determined. Usually it is apparent from the context or the type of the path which edge 
belongs to the path in multiple edges. We say a path is between the first and the last 
nodes of the list in G. We call the first and the last nodes endpoints of the path and all 
other nodes inner nodes. 

A path (or a cycle) in a directed graph is direction preserving if all its arrows point 
to one direction (o — — . . . — ^o). A directed graph is acyclic if it has no direction- 
preserving cycle. 

If in a direction-preserving path an arrow starts at a node j and an arrow points to a 
node i then j is an ancestor of i, and i a descendant of j. We use the notation an(i) for 
the set of all ancestors of i. 
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3. Independence model for ribbonless graphs 

Loopless mixed graphs. Graphs that wiU be discussed in this paper are subclasses of 
loopless mixed graphs. A mixed graph is a graph containing three types of edges denoted 
by arrows, arcs (two-headed arrows), and hues (sohd hues). Mixed graphs may have 
multiple edges of different types but do not have multiple edges of the same type. We do 

not distinguish between i j and j i or i < > j and j < > i, but we do distinguish 

between j — and i — >~j. Thus there are up to four edges as a multiple edge between 
any two nodes. A loopless mixed graph (LMG) is a mixed graph that does not contain 
any loops (a loop may be formed by a line, arrow, or arc). 

Some definitions for mixed graphs. For a mixed graph H, we keep the same termi- 
nology introduced before for directed and undirected graphs. We say that i is a neighbour 
of j if these are endpoints of a line, and i is a parent of j if there is an arrow from i 
to j. We also define that i is a spouse of j if these are endpoints of an arc. We use the 
notations Tie(j), pa(j), and sp{j) for the set of all neighbours, parents, and spouses of j 
respectively. 

In the cases of i — 7~j or i < > j we say that there is an arrowhead pointing to (at) j. 
A path (7 = iQ, in = *) is from j to i if jii is cither a line or an arrow from j to 
ii, and in-ii is either an arc or an arrow from in-i to i. 

A y -configuration or simply Vs is a path with three nodes and two edges. In a mixed 
graph the inner node of three Vs i — >- 1 — j , i < > t H — j , and i < > t < > j is a collider 
and the inner node of all other Vs is a non- collider node in the V or more generally in 
a path on which the V lies. We also call a V with collider or non-collider inner node a 
collider or non-collider V respectively. We may mention that a node is collider or non- 
collider without mentioning the V or path when this is apparent from the context. Notice 
that originally ([4]) and in most texts, the endpoints of a V are not adjacent whereas we 
do not use this restriction. 

Two paths (including Vs or edges) are called endpoint-identical if presence or lack of 
arrowheads pointing to endpoints on the path are the same in both. For example, i — 7~j, 

i k < > j, and i — >~k^ — I < > j are all endpoint-identical as there is an arrowhead 

pointing to j but there is no arrowhead pointing to i on the paths. 

Ribbonless graphs and its subclasses. The largest subclass of LMGs studied in 
this paper is the class of ribbonless graphs. 
A ribbon is a collider V (h,i,j) such that 

1. there is no endpoint-identical edge between h and j, i.e. there is no hj-avc in the 
case of h < > i < > j; there is no hj-line in the case of h — — j; and there is no 
arrow from h to j in the case of h — M ■< > j; 

2. i or a descendant of i is an endpoint of a line or on a direction-preserving cycle. 

A ribbonless graph (RG) is an LMG that docs not contain ribbons as induced subgraphs. 

Figure 1 illustrates ribbons {h, i, j). Figure 2(a) illustrates a graph containing a ribbon 
(/i, I, j). Figure 2(b) illustrates a ribbonless graph. Notice that {h, i, j) is not here a ribbon 
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since there is a line between h and j. 
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Figure 1. (a) A ribbon {h,i,j), where ne(i) — 0. (b) A ribbon {h,i,j). 
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Figure 2. (a) A grapli that is not ribbonless. (b) A ribbonless graph. 



The three classes of undirected graphs (UGs) (used for concentration graph models), 
bidirected graphs (BGs) (used for covariance graph models), and DAGs arc subclasses of 
RGs. SGs and AGs, which are studied in this paper, are also subclasses of RGs. Wc use 
the notations UQ, SQ, AG, UQ, BQ, and VAG for the set of all RGs, SGs, AGs, UGs, 
BGs, and DAGs respectively. The common feature of all these graphs is that these all 
entail independence models using the same so-called separation criterion, which is called 
m-separation and will be shortly defined. 

The m-separation criterion for RGs. The following definition was given in [10]. 

Let C be a subset of the node set V of an RG. A path is m-connecting given M and 
C if all its collider nodes are in C U an(C) and all its non-collider nodes are in M. For 
two other disjoint subsets of the node set A and B such that M ^ V \ AU B U C, we 
may just call the path m-connccting given C between A and B. We say AJ-mB \ C if 
there is no m-connecting path between A and B given C. 

Notice that the m-separation criterion induces an independence model J^m{G) on G 
byA±™B|C ^ e j;„(G). 



4. Marginalisation, conditioning, and stability 

Marginal and conditional independence models. Consider an independence model 
over a set V . For M a subset of V , the independence model J after marginalisation 
over M, denoted by a{J; M, 0), is the subset of J whose triples do not contain members 
of M, i.e. 

a{J; M, 0) = {{A, B \ D) e J : {AU B U D) D M = 0}. 
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Figure 3. (a) A directed acyclic graph Gi, by which it can be shown that the class of DAGs is 
not stable under marginalisation. ( ^ £ M.) (b) A directed acyclic graph G2, by which it can 
be shown that the class of DAGs is not stable under conditioning. ( [O] £ C .) 



One can observe that a{J; M, 0) is an independence model over V \ M . 
For a subset C of V , the independence model after conditioning on C, denoted by 
a(J;0,C), is 

a{J; 0,C) ^ {{A,B\D) : {A, B\DUC) e J and {AU BU D)r\C ^ 0}. 

One can also observe that a{J'; 0, C) is an independence model over V \ C. 
Combining these definitions, for disjoint subsets M and C of V, the independence 
model after marginalisation over M and conditioning on C is 

a{J] M, C) ^ {{A, B \ D) : {A, B \ D U C) e J and {AU B U D) n [M U C) ^ 0}, 

which is an independence model over V \ {M U C) . 

Notice here that a is a function from the set of independence models and two of their 
subsets to the set of independence models. Notice also that operations for marginalisation 
and conditioning commute. 

Marginalisation and conditioning in probability conform with marginalisation and 
conditioning for independence models. Consider a set N = V \ [M U C) and a collection 
of random variables {Xa)ai^N with joint density f(v\M) \ c- We associate an independence 
model to this density. It can be shown that if is the associated independence model 
to the collection of random variables {Xa)aGV with joint density fv then the associated 
independence model to {Xa)aeN with joint density f(y\M) \ c is oi{S , M, C). 

Stability under marginalisation and conditioning for RGs and its subclasses. 

Consider a family of graphs T. If, for every graph G = {V, E) E T and every disjoint 
subsets AI and C of V, there is a graph H E T such that Jm{H) = a{J„i{G);0,C) 
then 7" is stable under conditioning, and if there is a graph H Cz T such that Jm{H) = 
ct{Jm{G)] M, 0) then T is stable under marginalisation. We call T stable (under marginal- 
isation and conditioning) if there is a graph H & T such that Jm{H) = a{Jm{G); M, C). 

Notice that if the node set of such a graph H is N then = ^ \ (Af U C). 

We shall see that RGs, SGs, AGs, UGs, and BGs are stable. On the other hand, the 
class of DAGs is not stable. It can be shown that Gi in Fig. 3 is a DAG whose induced 
marginal independence model cannot be represented by a DAG and G2 is a DAG whose 
induced conditional independence model cannot be represented by a DAG. We leave the 
details as an exercise to the readers. 
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Stable mixed graphs. As the class of DAGs is not stable, we look for stable classes 
of graphs that include the class of DAGs as a subclass. In this paper we discuss three 
such types of graphs, namely RGs (as a modification of MCGs), SGs, and AGs, and 
specifically call these stable mixed graphs. We will see that in these graphs arcs are 
related to marginalisation and lines are related to conditioning. 

For the graph G2 & T for which J'^^ = a(Jm(Gi); M, C), we use the notation G2 = 
a-fiGi; M,C). For each type of stable mixed graphs, we later precisely define aj- with 
specific algorithms. We call aj- a generating function or more specifically a T -generating 
function. 

5. Ribbonless graphs 

MC graphs and ribbonless graphs. MCGs only contain the three desired types of 
edges. However, these are not loopless and, in addition, in MCGs a different separation 
criterion is used for inducing the independence model. However, from an MCG and by a 
minor modification one can generate an RG that induces the same independence model. 
This modification includes adding edges between pairs of nodes connected by a ribbon 
such that the generated edges preserve the arrowheads at the endpoints of the ribbon, 
and removing all the loops. Wc shall not go through the details of this modification in 
this paper. 

5.1. Generating ribbonless graphs 

A local algorithm to generate RGs from RGs. Here wc present an algorithm to 
generate an RG from a given RG and two subsets of its node set that will be marginalised 
over and conditioned on. This algorithm is local in the sense that, after determining the 
ancestor set of the conditioning set, it looks solely for all Vs in the graph and not for 
longer paths. Later in this section we will show that a graph generated by the algorithm 
is an RG and it induces the marginal and conditional independence model of the input 
graph by using m-separation. 

Suppose that H is an RG and consider M and C two disjoint subsets of the node set. 
There are 10 possible non-isomorphic Vs in an RG, displayed in Table 5.1. Notice that 
this table generates cndpoint-idcntical edges to the given Vs. Wc now define the following 
algorithm, derived from [16] and [5]. See also the appendix of [13]. 

Algorithm 1. aRciH; M,C): (Generating an RG from a ribbonless graph H) 
Start from H . 

Generate an endpoint identical edge between the endpoints of collider Vs with inner 
node in C U an(C) and non-collider Vs with inner node in M, i.e. generate an 
appropriate edge as in Table 5. 1 between the endpoints of every V with inner node 
in M or C U an(C) if the edge of the same type does not already exist. 
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Apply the previous step until no other edge can he generated. Then remove all nodes in 
M U C. 



Table 1. Types of edge induced by Vs with inner node in rn€M or s€CU an(C). 



generates 
generates 
generates 
generates 
generates 
generates 
generates 



i j 

i j 



8 
9 
10 



generates 
generates 
generates 



i j 



This method is a generahsation of the method used by [7], called moralisation, as a 
separation criterion on DAGs. Notice that the order of applying steps of Table 3.1 in 
Algorithm 1 is irrelevant since adding an edge does not destroy other Vs in the graph. 

Fig. 4 illustrates how to apply Algorithm 1 step by step to a DAG. We start from step 
1 of Table 5.1 and proceed step by step. We return to step 1 at the end if there are any 
applicable steps left. Since VAG C TZG, one can also use Algorithm 1 to generate an RG 
from a DAG. Notice that it is not enough to simply apply steps 1,4, and 10 of Table 5.1 
to a DAG. 



Global interpretation of the algorithm. The following lemma explains the global 
characteristics of the process of marginalisation and conditioning. 



Lemma 1. Let H be a ribbonless graph. There exists an edge between i and j in the rib- 
bonless graph aRoiH; M, C) if and only if there exists an endpoint-identical m-connecting 
path given M and C between i and j in H . 

Basic properties of a_RG- We show here that afja is an RG-generating function. 
Proposition 1. Graphs generated by Algorithm 1 are RGs. 

Notice that for every ribbonless graph H, it holds that aiia{H\ 0, 0) = H . 
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Figure 4. (a) A directed acyclic graph G, £ M and [O] £ C. (b) The generated graph after 
applying step 1 of the table, (c) The generated graph after applying step 4. (d) The generated 
graph after applying step 10. (e) The generated graph after applying step 8. (f) The generated 
RG from G. 



Surjectivity of a^G- The following result shows that the class of RGs is the exact 
class of graph that is generated after marginalisation and conditioning for DAGs. 

Proposition 2. The map urg ■ 'DAQ TZQ is surjective. 

5.2. Two necessary properties of RG-generating functions 

Here we establish the two important properties that a^a (or every generating function) 
must have. In short, it must be well-defined and it must generate a stable class of graphs. 

Well-definition of aRc- The following theorem shows that ana is well-defined. This 
means that instead of directly generating an RG we can split the nodes that we marginalise 
over and condition on into two parts, first generate the RG related to the first part, then 
from the generated RG generate the desired RG related the second part. 

Theorem 1. For a ribbonless graph H ~ {N,F) and disjoint subsets C, Ci, M, and 
Ml ofN, 

aRGiaRGiH; M, C); Ah,Ci) = aRciH; M U Mi, C U Ci). 

Stability of the graphs generated by aRo- Here we introduce the second important 
property that urg must have. This property is the core idea in defining RGs and in 



imsart-bj ver. 2011/05/20 file: sadeghi-arxiv.tex date: May 29, 2012 



10 



K. Sadeghi 



general stable mixed graphs. The modification applied by the function should generate 
a graph that induces the marginal and conditional independence model. 

Theorem 2. For a ribbonless graph H ~ {N, F) and disjoint subsets A, B, C , Ci, and 

M of N, 

ALmB\Ci in aHciH; M,C) ^ A ± ™S | C U Ci in H. 
Corollary 1. For a ribbonless graph H — {N,F) and M and C disjoint subsets of N , 

ai JrniH); M, C) - J„,{aRG{H; M, C)). 
Corollary 2. The class of RGs is stable. 

The following result has been implicitly discussed in the literature, e.g., see [2]. 
Corollary 3. The classes of UGs and BGs are stable. 

Proof. The result follows from the fact that, from UGs and BGs, Algorithm 1 generates 
UGs and BGs respectively. □ 

Example. Fig. 5 illustrates a DAG as well as two subsets M and C of its node set. 
Fig. 6(a) illustrates the generated ribbonless graph H using Algorithm 1 as well as two 
subsets Ml and Ci of its node set, and Fig. 6(b) illustrates the RG generated by the 
algorithm from H. 




Figure 5. A directed acyclic graph G with sixteen nodes, ^ G A/ and lOl £ C. 
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Figure 6. (a) The generated ribbonless graph H — aaciG, M, C) from the directed acychc graph 
G in Figure 5, ^ G Mi and [O] G Ci. (b) The generated ribbonless graph ORoiH, A'h,Gi) 
from H. 

For example, consider the graph ajioiH, Mi,Ci) in Fig. 6(b), and let A ~ {j}, B = 
{s} and C = {i, u). It is seen that v G an(C). We have that A is not m-separated from B 
given C since (j, i, v, t, s) is an m-connecting path between A and B given C. By Theorem 
2 we conclude that A is not m-separated from B given C U Ci . The same conclusion is 
made by observing m-connecting path {j, i, v, w, t, r, s) in H . 

6. Summary graphs 

Definition of summary graphs. A summary graph is a looplcss mixed graph H = 

{N, F) which contains no o o — o or o o < > o (arrowhead pointing to line) and 

no direction-preserving cycle as subgraph. Notice that there are also no multiple edges 
in SGs except multiple edges consisting of an arrow and an arc. 

Obviously the class of SGs is a subclass of RGs. Fig. 7 illustrates an SG and an RG 
that is not an SG. (Because of two reasons: existence of arrowheads pointing to lines and 
existence of a double edge containing line and arrow.) 



Figure 7. (a) An SG. (b) An RG that is not an SG. 

6.1. Generating summary graphs 

A local algorithm to generate SGs. We now present a local Algorithm (after de- 
termining the ancestor set of the conditioning set) to generate an SG from an SG. 

Algorithm 2. asciH; M^C): (Generating an SG from a summary graph H) 
Start from H. Label the nodes in an(C). 
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Figure 8. (a) A directed acyclic graph G, [O] G C. (b) The generated graph after applying step 
1, □ G an(C). (c) The generated SG after applying step 2. 

1. Apply Algorithm 1. 

2. Remove all edges (arrows or arcs) with arrowhead pointing to a node in an(C), and 
replace these by the edge with the arrowhead removed (line or arrow) if the edge 
does not already exist. 

Continually apply step 1 until it is not possible to apply the given step further before 
moving to the second step. 

Fig. 8 illustrates how to apply Algorithm 2 step by step to a DAG. Notice that as it 
is stated in the description of the algorithm the order of applying the steps does matter 
here. 

The map asG and its basic properties. For SGs we prove analogous results to 
those for RGs. 

Proposition 3. Graphs generated by Algorithm 2 are SGs. 

The map aRc.SG and its properties. Notice that step 1 of Algorithm 2 generates 
an RG before removing the nodes in C. Hence step 2 of the algorithm generates an SG 
from an RG and some extra nodes that are conditioned on. We denote these two steps 
by anG.SG- This shows that for generating RGs from SGs, an(C) is needed. 

Proposition 4. Let H = {N,E) be a ribbonless graph and M and C be subsets of N. 
It holds that asG{H; M, C) = aRG.sG{aB.G{H; M, C); an(C)). 

Surjectivity of asG- The following result shows that every member of SQ can be 
generated by a DAG after marginalisation and conditioning. 

Proposition 5. The map asG ■ T^AQ — !• SQ is surjective. 

6.2. Two necessary properties of SG-generating functions 

Here we express two important results that have been introduced for graphs generated 
by CiRG for graphs generated by asG- 
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Well-definition of aso- This property is analogous to the well-definition of ana as 
defined in the previous section. For a proof based on matrix representations of graphs 
and on properties of corresponding matrix operators, see [13]. 

Theorem 3. For a summary graph H — {N,F) and disjoint subsets C, Ci, M, and 
Ml ofN, 

asGiasaiH; M, C); Mi,Ci) - asaiH; MUMi,CUCi). 

Stability of tiie graphs generated by asG- We prove that analogous to RGs, graphs 
generated by asG induce the marginal and conditional independence model. This result 
can be implied from what was discussed in [13]. 

Theorem 4. For a summary graph H = [N, F) and disjoint subsets A, B, C , Ci, and 
M of N, 

A±,„,B\Ci in asGiH;M,C) ^ A±„^B\CLlCl in H. 
Corollary 4. For a summary graph H = {N,F) and M and C disjoint subsets of N , 

a{Jm{H); M, C) = JmiasciH; M, C)). 
Corollary 5. The class of SGs is stable. 

Example. Fig. 9(a) illustrates the generated SG from the DAG in Fig. 5 using Algo- 
rithm 2 as well as the two subsets Mi and Ci of its node set. Fig. 9(b) illustrates the SG 
generated by the algorithm from the SG in part (a). 




Figure 9. (a) The generated SG from the DAG in Figure 5, ^ G hh and [O] G Ci. 
(b) The generated SG from the SG in (a). 



7. Ancestral graphs 

An ancestral graph (AG) is a simple mixed graph that has the following properties for 
every node i: 
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1. i ^ an(pa(i) U sp(i)); 

2. If ne{i) ^ then pa(i) U sp(i) = 0. 

This means that there is no arrowhead pointing to a hne and there is no direction- 
preserving cycle, and there is no arc with one endpoint that is an ancestor of the other 
endpoint in the graph. 

AGs are obviously a subclass of SGs, and therefore RGs. Fig. 10 illustrates an SG that 
is not ancestral. (Because of an arc with one endpoint that is an ancestor of the other.) 



Figure 10. (a) An AG. (b) An SG that is not ancestral. 

7.1. Generating ancestral graphs 

A local algorithm to generate AGs. In [10] there is a method to generate AGs (in 
fact maximal AGs) globally by looking at the so-called inducing paths. Here we introduce 
an algorithm to generate AGs locally (after determining the ancestor set) by looking only 
for Vs after determining the ancestor set of the conditioning set. 

Algorithm 3. aAciH; M,C): (Generating an AG from an ancestral graph H) 
Start from H . 

1. Apply Algorithm 2. 

2. Generate respectively an arrow from j to i or an arc between i and j for V j — >~k < > i 
or V j < > k < > i when k G an(i) if the arrow or the arc does not already exist. 

3. Remove the arc between j and i in the case that j G an(i), and replace it by an 
arrow from j to i if the arrow does not already exist. 

Continually apply each step until it is not possible to apply the given step further before 
moving to the next step. 

Fig. 11 illustrates how to apply Algorithm 3 step by step to a DAG. 

The map uag and its basic properties. Basic properties of Algorithm 3 and its 
corresponding function are analogous to the basic properties of RGs and SGs. 

Proposition 6. Graphs generated by Algorithm 3 are AGs. 

As before we consider uag a-s a function from the set of AGs and two subsets of their 
node set to the set of AGs. 

Notice that by the extension of the generated AG to a maximal AG (as explained 
in [10]) the same maximal AG as that generated by the method explained in [10] is 
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Figure 11. (a) A directed acyclic graph G, ff ^ M. (b) The generated graph after applying 
step 1. (c) The generated graph after applying step 2 for V {/i, j, i). (d) The generated AG from 
G after applying step 3. 

generated, and hence these two graphs induce the same independence model. This also 
explains the global interpretation of the algorithm. Wc will not give the details in this 
paper. 

The map aso.AG and its properties. Notice that step 1 of Algorithm 3 generates 
an SG. Hence steps 2 and 3 of the algorithm generate an AG from an SG. We denote 
these two steps by asG.AG^ a function from SQ to AQ. 

Proposition 7. It holds that uag = ctSG.AG ° ceSG- 

Surjectivity of aAG- The following result shows that every member of AG can be 
generated by a DAG after marginalisation and conditioning. 

Proposition 8. The map aAG ■ 'DAG AG is surjective. 

Proof. The result follows from Proposition 5, the fact that AG ^ SG, and ii H E AG 
then asG.AG{H) = H. □ 

7.2. Two necessary properties of AG-generating functions 

Again we discuss the two important properties that we have proven for two other stable 
mixed graphs. 

Well-definition of uag- Well-definition of aAG is analogous to the well-definition of 
aRG and asG as defined in the previous sections. 

Theorem 5. For an ancestral graph H = {N,F) and disjoint subsets C, Ci, M, and 



Ml ofN, 



aAGiaAG{H; Af, C); A'h,Ci) = aAG{H; MUMi,CUCi). 
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Stability of the graphs generated by uag- Analogous to RGs and SGs, graphs 
generated by uag induce marginal and conditional independence models. An analogous 
result was proven in [10] for maximal AGs that were generated in that paper. 

Theorem 6. For an ancestral graph H ~ {N,F) and disjoint subsets A, B, C, Ci, 
and M of N, 

ALmB\Ci maAG{H]M,C) ^ A±„iB\CUCi m H. 

Corollary 6. [10] For an ancestral graph H ~ {N, F) and AI and C disjoint subsets 
ofN, 

a{Jra{H)- M, C) = J,n{aAG{H; M, C)). 
Corollary 7. The class of AGs is stable. 

Example. Fig. 12(a) illustrates the AG generated from the DAG in Figure 5. Fig. 
12(b) illustrates the AG generated by the algorithm from the AG in part (a). 




Figure 12. (a) The generated AG from the DAG in Figure 5, ^ £ Mi and [O] £ Ci. (b) The 
generated AG from the AG in (a). 

8. The relationship between different types of stable 
mixed graphs 

Thus far, we have defined RGs (as a modification of MCGs), SGs, and AGs, and intro- 
duced algorithms to generate each of these from a graph of the same class or a DAG, and 
some algorithms that act between these classes. Despite the similarities of these defini- 
tions and generating algorithms of these different classes, as well as the parallel theory 
developed for these, it is of interest to investigate the exact relationship between these 
types of graphs. 
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Corresponding stable mixed graphs. When one starts from a DAG and generates 
different types of stable mixed graphs after marginalisation over and conditioning on 
two specific subsets of the node set of the DAG, the generated graphs must induce the 
same independence models. This leads us to the definition of corresponding stable mixed 
graphs. For a directed acyclic graph G and two disjoint subsets of its node set M and 
C, graphs aRciG] M,C), asciG; M,G), and aAciG; M,G) are called respectively the 
corresponding RG, SG, and AG. 

We observe that the corresponding RGs, SGs, and AGs of a DAG induce the same 
independence model. This fact, without being formulated in this way, was discussed in 
all three papers that define these graphs [5; 10; 13]. 

Proposition 9. For a directed acyclic graph G = {V, E) and disjoint subsets C and M 



As it was shown, in SGs and AGs there are extra properties regarding the structure 
of the graph. We know that AG C SG C TZG- The corresponding AG to an SG can 
be generated by asG.AG as outlined in Proposition 7. However, we cannot generate 
the corresponding SG to an RG by only knowing the RG and not the DAG (or the 



— o and o < > o respectively. This is also true for AGs instead of SGs. 

It is possible, however, to introduce an algorithm to generate SGs that induce the 
same independence model as the given RGs, by removing arrowheads pointing to a line 
or a node that is an ancestor of a node that is the endpoint of a line. 

We have also seen that the image of generating functions is big enough to cover all 
graphs included in the set of the related type of stable mixed graphs, since the generating 
functions are surjective. On the other hand, it is easy to show that generating functions 
are not injective. Therefore, the relationship between the three types of stable mixed 
graphs is summarised by the diagram in Fig. 13, in which one can only move towards 
the directions that arrows show. 



ofV, 



J,n{aRG{G; M, C)) = J,n{asG{G; M, C)) = J,n{aAG{G; M, G)). 



Proof. The result follows from Corollaries 1, 4, and 6. 



□ 





Figure 13. The relationship between T>AQ, TZQ, SQ, and AG- 
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9. Discussion on the use of different types of stable 
mixed graphs 

By what wc discussed, if G is a DAG with latent variables M and selection variables C 
then stable mixed graphs are a class of graphs that represent the independence model 
implied among the remaining variables, conditional on the selection variables. However, 
each of the three types has been used in different contexts and for different purposes. 

Why MCGs or RGs? MCGs have been introduced in order to straightforwardly deal 
with the problem of finding a class of graphs that is closed under marginalisation and 
conditioning by a simple process of deriving these from DAGs. In fact, the class of MCGs 
is much larger than what one really needs for representing independence models after 
marginalisation and conditioning. We have noted that only MCGs that are ribbonless 
can be generated this way. 

Why SGs? The main goal of defining SGs is to trace the effects after marginalisation 
and conditioning, as will be explained shortly in this section. By using binary matrix 
representations of graphs, called edge matrices, and corresponding matrix operators [17], 
the edge matrix of a SG is obtained. It contains three types of edge matrices: those for 
solid lines, dashed lines (corresponding to arcs), and for arrows. In the family of joint 
Gaussian distributions, solid lines in concentration graphs correspond to concentration 
matrices, dashed lines in covariance graphs to covariance matrices and arrows to equation 
parameters in structural equation models. 

SGs are used when the generating DAG is known. Despite knowledge on the structure 
of the generating DAG, SGs are still of interest in at least three situations: (1) For models 
with large number of unobserved and selection variables; and (2) for the comparison of 
models when one of them has unobserved or selection variables that are a subset of the 
unobserved or selection variables of the other; (3) for detecting some types of confounding 
as shown in [15] and as described briefly later. 

Why AGs? The main goal of defining AGs is to represent and parametrise sets of 
distributions obeying Markov properties. Even though, we discussed the class of AGs in 
this paper to sustain a parallel theory to RGs and SGs, the class of maximal AGs possess 
some desired properties that AGs do not. These include the fact that under the Gaussian 
path diagram parametrisation the maximal AG only implies independence constraints, 
while a general AG implies other types of constraints. We will give a short discussion on 
maximality in this section. Maximal AGs are the simplest structures that capture the 
modified independence model, and are also of interest when the generating DAG is not 
known, but a set of conditional independencies is known. In the Gaussian case maximal 
AGs are identified. In contrast to DAG models with hidden variable, the models are 
curved exponential families [10], and conditional fitting algorithm for maximum likelihood 
estimation exists [3]. 
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Maximal stable mixed graph. A graph G is called maximal if by adding any edge 
to G the independence model induced by _Lm changes (gets smaller). Therefore, in 
maximal graphs, every missing edge corresponds to at least one independence statement 
in the induced independence model. This leads to validity of a so-called pairwise Markov 
property. 

In [10] maximality of the subclass of AGs was studied. This result also holds for RGs 
and says that a ribbonless graph H is maximal if and only if H docs not contain any 
primitive inducing paths, which arc paths of form {j, qi,q2, . . . ,qp,i), on which i j and 
for every n, 1 < n < p, g„ is a collider on the path and g„ G an({i} U {j}). We shall not 
give the details in this paper. 

Therefore, to generate a maximal stable mixed graph from a stable mixed graph one 
should repeatedly generate arrows from j to i for primitive inducing paths between non- 
adjacent i and j where there is no arrowhead pointing to j, and generate arcs between i 
and j for primitive inducing paths between non-adjacent i and j where there are arrow- 
heads pointing to i and j. Notice that by applying this algorithm after the generating 
algorithms one can generate a maximal AG, SG, or RG. 

As discussed, maximal AGs possess many desired properties that AGs do not. For 
SGs, it is conjectured that maximal SGs possess the same statistical properties that 
both maximal AGs and SGs do possess. To show this, further work is needed. 

The structure of difTerent types of stable mixed graphs. If we suppose that sta- 
ble mixed graphs are only used to represent the independence model after marginalisation 
and conditioning then we can consider all types as equally appropriate. The question then 
will be reduced to how simple or fast generating a type of graph is. We have seen that 
AGs have the simplest structure among the three types of stable mixed graphs, and RGs 
are the most complex. Therefore, as we have also seen, it is more complex to generate 
an AG than to generate an SG, and to generate an SG than to generate an RG. On the 
other hand, the simpler structure allows a faster way of checking independence state- 
ments. Hence it is a tradeoff that depends on the relative size of the marginalisation and 
conditioning sets in graphs. 

When generating stable mixed graphs from DAGs, one always loses some information 
in order to obtain a simpler structure in stable mixed graphs. RGs have lost the least 
information among the three types of stable mixed graphs, while AGs the most. Here we 
discuss the lost information in the context of regression analysis. 

Multivariate regression and stable mixed graphs. The problem of constructing 
stable mixed graphs was originally posed by [16] in the context of multivariate statistics 
based on regression analysis. In such literature the DAG model is defined by sequences 
of univariate recursive regressions, called a linear triangular system by [14], i.e. for i = 
1, . . . , Jat — 1, each single response variable Yi is regressed on i^pa(i) , where the parents of i 
arc a subset of 1, . . . , dN}- Linear triangular systems can be written as AY = e, where 
A is an upper-triangular matrix with unit diagonal elements, and e is a vector of zero 
mean and uncorrclated random variables, called residuals. Here the non-zero regression 
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coefficient of Yi on Yj can be attached the arrow from j to i in the DAG and is called 
the direct effect of Yj on Yi\ see [1]. 

In particular, for linear triangular systems, RGs alert to distortions due to so-called 
over-conditioning via multiple edges consisting of a line and an arrow. Over-conditioning 
arises by conditioning on a variable that is a response of two variables, one of which itself 
is a response to the other one. 

For example, in Fig. 14, the generating process is given by three linear equations, 

Yi^pY2 + 5Y^ + ei, r2=7>3 + e2, >3 = £3, 

where each residual has mean zero and is uncorrelated with the explanatory variables 
on the right-hand side of the equation. 




(a) (b) (c) 

Figure 14. (a) A directed acyclic graph G with node 1 to be conditioned on. (b) The RG 
generated from G. (c) The SG or AG generated from G. 

By conditioning on Yi, the conditional dependence of I2 on only ¥3 is obtained, which 
consists of the direct effect 7 and an indirect effect of I2 on Y3 via Yi . This may be seen 
by direct calculation, assuming that the residuals have a Gaussian distribution, which 
leads to 

E(Y2 I Ya) = (7 - {(1 - 7')/(l - P?3)}/3pi3)>^2, where P13 = 5 + h- 

Thus, the direct effect 7 is distorted by — {(1 — 7^)/(l — /Oi3)}/3/3i3- The potential presence 
of this distortion is represented in (b) by the addition of an arrow. 

In addition, the existence of multiple edges with an arrow and an arc, and arcs with 
one endpoint ancestor of the other, which are not permissible in AGs, respectively alerts 
distortions due to so-called direct and indirect confounding. 




(a) (b) (c) 

Figure 15. (a) A directed acyclic graph G with node 3 to be marginalised over, (b) The SG 
generated from G. (c) The AG generated from G. 

With the same generating process as explained for Fig. 14, by integrating out I3 in 
Fig. 15, the conditional dependence of Yi on only Y2 is obtained, which consists of the 
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direct effect /3 and an indirect effect of Yi on I2 via Y3 . This leads to 

E{Yi\Y2)^{P + S-/)Y2. 

Thus, the direct effect (3 is distorted by S-f. The potential presence of this distortion 
is represented in (b) by the addition of an arc. This example indicates a distortion 
due to direct confounding; see [15]. Indirect confounding was also studied in [15] for 
marginalising only over a full set of background variables and also in [13] more generally 
relating SGs to corresponding maximal AGs. 

Appendix: Proofs 

Here we present the proof of lemmas, propositions, and theorems of this paper, but first 
we introduce some observations that are used in our proofs as the following lemmas. 

Lemma 2. If i £ an(j) in anciH] M,C) then in H one of the following holds: 1) 
i G an(j); 2) i or a descendant of i is the endpoint of a line; 3) i £ an(C). 

Proof. We know that there is a direction-preserving path tt = {i = iq, ii, . . . , = j) in 
aRoiH', M,C). Consider the ioii-edge. By Lemma 1 in H given M and C there is an 
m-connecting path between io and ii, on which there is no arrowhead pointing to io. One 
can observe that if this path is not a direction-preserving path then one of the following 
holds: (1) io is an ancestor of a collider node on the path, which is in C U an(C). Hence 

i S an(C); (2) io is the endpoint of a line or an ancestor of a node that is the endpoint 
of a line on the path. If (1) or (2) holds then we are done, hence assume that ig £ an(ii). 
By the same argument and by induction along the nodes of tt we conclude the result. □ 

Lemma 3. For i and j outside M U C, if i G an(j) in H then one of the following 
holds: 1) i e an(j) in aRG{H;M,C); 2) i £ an(C) in H. 

Proof. We know that there is a direction-preserving path tt = {i = iq, ii, . . . , = j) in 
H. Consider the io«i-edge. We now have three cases: 1) If ii G C then i G an(C) in H 
and we arc done. 2) If ii G M then Algorithm 1 generates an arrow from io to i2- 3) If 

ii ^ M U C then zq G an(i2) in a^ciH] ^J, C). By the same argument and by induction 
along the nodes of tt wc conclude the result. □ 

The following Lemma deals with the concatenation of m-connecting paths. We shall 
not give the details of the proof here; see [11]. 

Lemma 4. In an RG, suppose that given M and C there are m-connecting paths {i = 
io,iiT ■ ■ ,in, h) between i and h and {j ~ ji3,ji, ■ ■ ■ ,jrm h) between h and j . In this case 
there is an m-connecting path given M and C between i and j if one of the following 
holds: 
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al) {in, h,jm) is collider and h £ C U an(C); 

a2) iji — jm, with CLTTOwhead pointing to h on the i^h-edge and h £ C U tin(C)j' 
bl) {in, h,jm) is non-collider and h e M ; 

b2) in — jm with no arrowhead pointing to h on the inh-edge and h G A/. 

cl) {in,h,jm) is collider and h or a descendant of h is the endpoint of a line or a 

direction-preserving cycle; 
c2) in ~ jm with arrowhead pointing to h on the inh-edge and h or a descendant of h is 

the endpoint of a line or a direction-preserving cycle. 

Proof of Proposition 1 . Graphs generated by Algorithm 1 have obviously three de- 
sired types of edges and are loopless. 

Now suppose, for contradiction, that there is a ribbon {i,h,j) in a generated graph 
aRG{H; M,C). By Lemma 1 in H given M and C there are m-connecting paths tti = 
{i = io, in, h) between i and h and tt2 = {j = jo,ji, ■ ■ ■ ,jm, h) between h and 

j such that there are arrowheads at h on both ink- and jmh-edges. (Notice that it is 
possible that in = jm and it is also possible that in = i or jm = j in .) 

We also know that, in auaiH; M,C), the node h is the endpoint of a line or on a 
direction-preserving cycle or there is a direction- preserving path tt = {h ~ Hq, hi, . . . , hp = 
k) from h to k such that k is the endpoint of a line or on a direction-preserving cycle. 
Now we consider two cases. In case I we suppose that such a tt does not exist and in case 
II we suppose that such a tt exists. 

Case I. In case I.l we suppose that h is the endpoint of a line and in case 1.2 we 
suppose that h is on a direction-preserving cycle. 

Case I.l. Suppose that h is the endpoint of an hl-line in anciH; M, C). By Lemma 
1 in 77 given M and C there is an m-connecting path between h and I, on which there 
is no arrowhead pointing to h or I. One can observe that h is an ancestor of either (1) a 
collider node on the path or (2) a node that is the endpoint of a line on the path. Thus 
we have the two following cases: 

(1) If h is an ancestor of a collider node t then h S an(C) in H since t S C U an(C). 
Hence, by Lemma 4(a), there is an m-connecting path given M and C between i and j 
in H. 

(2) If h is an ancestor of a node that is the endpoint of a line on the path then by 
Lemma 4(c) there is an m-connecting path given M and C between i and j in H . 

By Lemma 1 both cases imply that i ~ j in aaciH; M, C) and the zj-edge is endpoint- 
identical to the m-connecting path. Therefore, {i,h,j) is not a ribbon, a contradiction. 

Case 1.2. Suppose that /i is on a direction-preserving cycle in anciH; M,C). By 
Lemma 2 in H one of the following holds: (1) h S an(/i); (2) h oi a descendant of h is the 
endpoint of a line; (3) h g an(C). Cases (2) and (3) lead to contradiction as explained 
in case I.l. Therefore, suppose that ft, is on a direction- preserving cycle in H. This by 
Lemma 4(c) implies that there is an m-connecting path given M and C between i and j 
in H, which implies that {i, h, j) is not a ribbon. This is a contradiction. 

Case 11. By Lemma 2 in if one of the following holds: (1) h G an(fc); (2) h oi & 
descendant of h is the endpoint of a line; (3) h G an(C). Cases (2) and (3) lead to 
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contradiction as explained in case I.l. Hence it holds that h G an(fc) in H. This together 
with the same argument as that of case I (for k instead of h) leads to a contradiction. □ 

Proof of Lemma 1. (^) If an edge between i and j in aRciH; M,C) does not exist 
in H then it has been generated by certain intermediate graphs that have each been 
generated by adding one edge to the previous graph by one of the steps of Table 1. We 
denote these graphs by the sequence {H = Hq, Hi, . . . , Hn,af(G{H; M, C)), where Hn is 
the last step before removing M and C. 

We prove by reverse induction on p that in all Hp, < p < n, between i and j there 
exists a path on which non-collider inner nodes are in M and collider inner nodes or their 
descendants are either in C or the endpoint of a line. For p = n, there is obviously an 
edge between i and j. We show that if there is such a path in H^ then we can find the 
same type of path between i and j in Hr-i- 

If all edges along the path exist in Hr-i then we should check that a collider node 
that is an ancestor of a member of C or an ancestor of a node that is the endpoint of a 
line in Hr is an ancestor of a member of C or an ancestor of a node that is the endpoint 
of a line in Hr-i- If an arrow has been generated along the direction-preserving path 
in Hr then it has been generated by the Vs {i',m,j') of the first three steps or the V 
(i', s,j') of step 8 of Table 1. If it is step 1 then we can replace the I'j'-arrow by (i' , m, j') 
to obtain a direction-preserving path. If it is steps 2 or 3 then node j' is the endpoint of 
a line and we are done. If it is step 8 then, since s G C U an(C), the inner node of the V 
is in an(C) and we are done. 

Thus suppose that an i'j'-edge along the m-connecting path is the edge that has been 
generated by this step. This has been generated by one of Vs of Table 1. Since in all cases 
the V is endpoint-identical to the i'j'-edge, and since all inner nodes of the non-collider 
Vs are in M and all inner nodes of the collider Vs are in C U an(C), by placing the V 
instead of the i'j'-edge on the path, we still get a path whose non-collider inner nodes 
arc in M and either whose collider inner nodes are in C U an(C) or whose collider nodes 
or a descendant of them are the endpoint of a line, as required. 

Therefore, by reverse induction, there exists a path, as described above, in H. However, 
since H is ribbonless, the path cannot contain a collider V {i',h,j') such that h or a 
descendant of h is the endpoint of a line unless i' ^ j' and the i'j'-edge is endpoint- 
identical to {i', h,j'). In this case, the i'j'-edge can be used instead of (i' , h,j') and, by 
induction, we obtain an m-connecting path given M and C between i and j. 

The fact that in Table 1 the Vs are endpoint-identical to the generated i'j'-edges 
implies that all discussed paths in each Hp are endpoint-identical. 

(<J=) Suppose that there is an m-connecting path tt given M and C between i and j 
in Hk, < k < n — 1. We prove as long as r > if there is an m-connecting path given 
M and C between i and j in Hk with r inner nodes then there is an m-connecting path 
given M and C between i and j in H^+i with r — 1 inner nodes. By induction we will 
finally obtain an m-connecting path between i and j without inner nodes, i.e. an edge 
between j and i. 

Consider an m-connecting path given M and C between i and j in Hj- with r > 
inner nodes. Consider an arbitrary inner node on the path. If this node is collider then 
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one of the Vs 8, 9, or 10 of Table 1 is employed to generate an edge between the endpoints 
of the V in Hk+i- Since the generated edge is endpoint-identical to the V, one can use 
the generated edge instead of the V to obtain an m-connecting path with r — 1 inner 
nodes. If the arbitrary node is non-collider then one of the other Vs of Table 1 is used. 

It is easy to check that the generated edges are endpoint-identical to the m-connecting 
paths in the final graph. This implies the result. □ 

Proof of Proposition 2. Let H = {Ni,Fi) S TZQ. We generate a directed graph G = 
(V, E) from H as follows: We leave arrows that are not on any direction-preserving cycle 
unchanged. For direction-preserving cycles, instead of one arbitrary arrow from j to i on 
the cycle we place j — h- |0] -< — ^ — T^i, where G M and |0] S C, and leave all other 
arrows unchanged. Instead of an arc between j and i we place a V between j and i with 
inner source node in M. Instead of a line between j and i we place a V between j and i 
with inner collider node in C. The graph G is obviously a directed graph. Furthermore, 
all newly generated nodes have degree 2 and the direction of arrows changes on them, 
hence these cannot be on any direction-preserving cycle. In addition, if i and j are in Ni 
and i ~ j in G then i G pa(j) or j € pa(i) in H. Therefore, the existence of a direction- 
preserving cycle in G implies the existence of the same direction-preserving cycle in H. 
But by the nature of the construction of G we know that direction-preserving cycles in 
H do not make direction-preserving cycles in G, hence G is acyclic. 

Wc should prove that aRG{G;M,C) = H. Let aRG{G]M,C) = (iV2,-F2)- Obviously 
A^i = N2- Suppose that i ^ j {j e pa(i), j G sp(i), or j e ne(i)) in H. Therefore, we 
have the active alternating path or one of the Vs between i and j that by Algorithm 1 
forms exactly the same type of edge in aRciG; M, C). 

Conversely, suppose that i ^ j {j 6 pa(i), j G sp(i), or j e nc(i)) in anciG; M, C). By 
Lemma 1 we know that there is an endpoint-identical m-connecting path given M and C 
in G. Consider a shortest endpoint-identical m-connecting path tt. Since in G there is no 
transition node in M, tt is active alternating with respect to M and C U an(G). If tt has 
no collider node in an(G) \ G then by the nature of the construction of G we know that it 
has two edges (if both endpoints are children or parents) or three (if it is from j to i on a 
direction-preserving cycle) and that it has been generated by an edge (arrow, arc, or line) 
in H. Suppose, for contradiction, that there is a collider node i € an(G) \ G on tt. We 
have that i € Ni, and by the process of generating a DAG explained here, the only place 
that a node in G has been generated is by a line or an arrow on a direction-preserving 
cycle in H. Therefore, i € a,n{k) for a node k that is the endpoint of a line or is on a 
direction-preserving cycle. Hence H contains a ribbon, or the endpoints of the collider V 
with i as inner node are adjacent by an endpoint-identical edge. The former contradicts 
that H is ribbonless, and the latter contradicts that tt is shortest. □ 

Proof of Theorem 1. (^) If there is an ij-edge in ajic{(XRG{H\ M, G); Mi, Gi) then 
by Lemma 1, there is an m-connecting path tt = = iq, ii, . . . , in-i, in = j) between i 
and j given Mi and Gi in aRciH; M, G) that is endpoint-identical to the edge. 

For the V (i, ii, 12) on tt, again by Lemma 1, given M and G there are m-connecting 
paths TTi between i and ii and tt2 between ii and i2 in H. These paths can be considered 
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m-connecting given M U Mi and C U Ci and are endpoint-identical to the edges. This 
imphes that if ii is colhder (or non-colhder) on tt then on the concatenation of tti and 
7r2 it remains colhder (or non-cohider), or that there is an arrowhead pointing to it (or 
no arrowhead pointing to it) from a joint node on tti and 7r2. We know that if ii is non- 
colhder then it is in Mi and if it is collider then it is in Ci U an(Ci) in anciH; M, C). 
If ii S an(Ci) in aRciH; M,C) then, by Lemma 2, one of the following holds in H: 1) 
it is in an(Ci); 2) it is in an(C); 3) it is the endpoint of line or an ancestor of a node 
that is the endpoint of a line. Therefore, by Lemma 4, there is an m-connecting path 
between i and Z2 given M U Mi and C U Ci in H, which is endpoint-identical to the V. 
By induction along tt there is an m-connecting path between i and j given AI U Mi and 
C U Ci in H, which is endpoint-identical to the ij-edge. Therefore, by Lemma 1 there is 
the same type of ij-edge in aRoiH; M U Mi, C U Ci). 

(<;=) If there is an edge between i and j in aRciH; M U Mi,C U Ci) then, by Lemma 
1, there is an m-connecting path tt given M U Mi and C U Ci in that is endpoint- 
identical to the zj-edge. All inner nodes of tt are either in M U C U an(C) or Mi U Ci U 
an(Ci). Therefore, tt can be partitioned into m-connecting subpaths given M and C and 
single nodes, where the endpoints of subpaths and single nodes are in Mi U Ci U an(Ci). 
Therefore, by Lemma 1, in aRG{H] M, C) the endpoints of each of the discussed subpaths 
of TT are connected by an edge that is endpoint-identical to the subpath. 

In addition, for each collider V (Z, fc, /i), where k £ an(Ci) in H and by Lemma 3, one 
of the following holds: 1) k G an(C) in H; 2) k e an(Ci) in aRG{H;M,C). Case (1) 
implies that there is an endpoint-identical Ih-edge to the V in aRciH; M, C), which can 
be used instead of {I, k, h) to generate an m-connecting path. 

Hence in andH; M,C) there is an m-connecting path given Mi and Ci between i 
and j, which is endpoint-identical to tt. Therefore, again by Lemma 1, there is the same 
type of ij-edge in aBG{aRG{H] M, C); Mi, Ci). □ 

Proof of Theorem 2. To prove the result, it is enough to show that, between nodes i 
and j outside C U Ci U M, there is an m-connecting path given C U Ci in H if and only 
if there is an m-connecting path given Ci in anciH; M, C). 

(=^) Suppose that between i and j there is an m-connecting path given C U Ci in 
H. This path can be partitioned into m-connecting subpaths given C and single nodes, 
where the endpoints of subpaths and single nodes are cohiders in Ci Uan(Ci). In addition, 
for each collider V {I, k, h), where k € an(Ci) in H and by Lemma 3, one of the following 
holds: 1) k G an(C) in H; 2) k G an(Ci) in aRG{H;0,C). Case (1) implies that there 
is an endpoint-identical //i-edge to the V in aua^H; 0, C), which can be used instead of 
{I, k, h) to generate an m-connecting path. 

Hence by Lemma 1, in aRoiH; 0, C) there is an m-connecting path given Ci between i 
and j. The inner non-collider nodes on this path are either in M or in N\ (MUCUCi). On 
this path there are subpaths with only non-collider inner nodes in M, i.e. m-connecting 
subpaths given M and 0. By Theorem 1 after marginalisation over M, aRciH; M, C) is 
obtained, in which, by Lemma 1, the endpoints of each of such subpaths are connected 
by an edge that is endpoint-identical to the subpath. In addition, if a collider node is in 
an(Ci) in aRciH; 0, C), then by Lemma 3 (since the conditioning set is empty and case 
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(2) of the lemma does not hold), the collider node remams in an(Ci) in aiia{H; M, C). 
Therefore, between i and j there is an m-connecting path given Ci in aadH] M, C). 

(<J=) Suppose that between i and j there is an 771-connccting path tt given Ci in 
oiRQ^H; M,C). By Lemma 1, for each edge of tt, there is an endpoint-identical m- 
connecting path given M and C in i?, which is obviously an m-connecting path given C . 
On the concatenation of these paths and for the cndpoints of the paths we have the two 
following cases: 1) When the endpoints are non-collider or there is no arrowhead at them 
on the concatenation, they are non-collider on tt and therefore outside M U C U Ci; 2) 
When the endpoints are collider or there is an arrowhead at them, they are collider on 
TT and therefore in Ci U an(Ci). Therefore, by Lemma 4, there is an m-connccting path 
given C U Ci in iJ. □ 

Proof of Proposition 3. Firstly, the graph generated by the algorithm has only the 
three desired types of edges and no multiple edge of the same type; therefore, it generates 
a mixed graph. 

Let H be the input summary graph and Hq = aRc^H; M, C) be the generated graph 
after applying step 1 of Algorithm 2. Here in part I we prove that there is no arrowhead 
pointing to lines, and in part II we prove that there is no direction-preserving cycle in 
the generated graph. 

Part I. If, for contradiction, there is an arrowhead at i (on an ifc-cdgc) and i is the 
endpoint of an ij-line in the generated graph then we have the following cases: 1) The 
ij-line is an arrow from i to j in Hq; 2) the jj-line is an arrow from j to i in Hq] 3) the 
ij-line is an arc in Hq] 4) the ifc-edge is an arrow from fc to i in the generated graph and 
an arc in Hq; 5) the ki- and ij-edges are the same in i?o- 

1) We have that j G an(C) in H . Hence by Lemma 2 we have one of the two following 
cases in H: a) i or a descendant of i is the endpoint of a line, which, since H is a. 
summary graph, is a contradiction; h) i £ an(C), which by the algorithm implies that in 
the generated graph there is no arrowhead at i on the i/c-edge, again a contradiction. 

2,3) We have that i G an(C) in H , which by the algorithm implies that in the generated 
graph there is no arrowhead at i on the ifc-edge, a contradiction. 

4,5) We have that {k,i,j) still has an arrowhead pointing to a line in Hq. By Lemma 

1 there is an arrowhead at i in H, hence the line has been generated by the algorithm. 
Again by Lemma 1 we observe that one of the following holds: a) i or a descendant 
of i is the endpoint of a line, which, since iJ is a summary graph, is a contradiction; 
h) i G an(C), which by the algorithm implies that in the generated graph there is no 
arrowhead pointing to i on the ik-edge, again a contradiction. 

Part II. There is also no direction-preserving cycles in the generated graph: If, for 
contradiction, there is a direction-preserving cycle in the generated graph then at least 
one arrow, say from k to should be generated by the algorithm since there is no 
direction-preserving cycle in H. This arrow can be generated either by step 1, or by step 

2 as an arc replaced by an arrow. If the fc^-arrow is generated by step 2 then we have 
that k £ an(C) in H. Therefore, there are no arrowheads pointing to k in the generated 
graph, which means that k cannot be on a direction-preserving cycle, a contradiction. 
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Therefore, we can assume that the direction-preserving cycle exists in Hq. Since there 
are no arrowheads pointing to hnes in H and H does not contain a direction-preserving 
cycle, Lemma 2 implies that a node (and therefore all nodes) of the cycle are in an(C). 
Hence the arrows turn into lines in the generated graph, a contradiction. □ 

Proof of Proposition 5. Wc know that SQ C TZG- Notice that iJ is a summary 
graph. We know that, for the generated directed acyclic graph G, explained in (a), 
<^rg{G; M,G) = H. Suppose, for contradiction, that step 2 of Algorithm 2 changes 
the graph. Thus a node with an arrowhead pointing to is in an(C), which implies that it 
is an ancestor of a node that is the endpoint of a line or on a direction-preserving cycle 
in H, a contradiction. Therefore, asaiG; M,G) = H. □ 

Proof of Theorem 3. We show that there is an edge between i and j in asaiH; M U 
Mi,C U Ci) if and only if there is the same type of edge in asG{oiSG{H] M, G); Mi,Gi). 
For this purpose for summary graphs it is enough to prove that there is an edge between 
i and j with arrowhead pointing to j in asaiH; M U Afi, C U Ci) if and only if there is 
an edge between i and j with arrowhead pointing to j in asoictsaiH; M, C); Mi, Ci). 

(^) Suppose that in asaiH; M U Mi,C U Ci) there is an ij-edge with arrowhead 
pointing to j. We have that j ^ an(C U Ci) in and in aRa{H; M U Mi, C U Ci) there 
is an ij-edge with arrowhead pointing to j. By Lemma 1 there is an m-connecting path 
between i and j given M U Mi and C U Ci in iJ with arrowhead pointing to j . By what 
we showed before in the proof of Theorem 1 in aaciH; M, C) there is an m-connecting 
path TT between i and j given Mi and Ci with arrowhead pointing to j. 

Now notice that by step 2 of Algorithm 2 non-collider nodes remain non-collider. In 
addition, if a collider V (/i, fc, I) on tt turns into non-collider then k G an(C) and therefore 
by step 1 of the algorithm there is an endpoint-identical hl-edge that can be used instead 
of the V to generate an m-connecting path. Moreover, if the collider node k is in an(Ci), 
and on the direction-preserving path an arrow turns into a line then k S an(C) in H and 
once again there is an hl-edge to be used instead of the V to establish an m-connecting 
path. Therefore, there is an m-connecting path between i and j given Mi and Ci in 
otsa{H; M, C). We also have that since j ^ an(C) in H, there is an arrowhead pointing 
j on the path. 

Now by Lemma 1 in aRcic^saiH; Af, G); Mi, Ci) there is an ij-edge with arrowhead 
pointing j. In addition, in asaiH; M, G), j ^ an(Ci): This is because if, for contradiction, 
j e an(Ci) in asa{H; M, G) then, in aRciH; M, C), j € an(C U Ci). This by Lemma 2 
and the fact that H is a summary graph implies that j £ an(CUCi) in H, a contradiction. 

Therefore, since in asa{H;M,G), j ^ an(Ci), there is an arrowhead pointing j on 
the ij-edge in aso{asa{H;M, C);Mi,Ci). 

(<^) Suppose that in asa{oisc(H; M, C); Mi, Ci) there is an ij-cdge with arrowhead 
pointing to j. This implies that j ^ an(Ci) u\asc{H;M,G). In anaictsaiH; M,G); Mi,Gi) 
there is also an ij-edge with arrowhead pointing to j. By Lemma 1, in asaiH; M^G) 
there is an m-connecting path tt given Mi and Ci between i an j with arrowhead pointing 
to j on the path. This implies that j ^ an(C) in H. 
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By step 2 of the algorithm, coUider nodes on tt m asaiH; M,C) are colHdcrs in 
aRoiH; M, C). In addition, if a non-colhder V (/i, fe, I) on tt is coUider in aRciH; M, C) 
then fc G an(C) in 77 and therefore by step 1 of the algorithm there is an endpoint- 
identical hl-edge that can be used instead of the V to generate an m-connecting path in 
aRG{H; M, C). Moreover, if the collider node k on tt is in an(Ci), and on the direction- 
preserving path an arrow is an arc in anaiH; M, C) then the collider node is in an(C) 
in H and once again there is an hl-edge to be used instead of the V to establish an 
m-connecting path. Therefore, there is an m-connecting path between i and j given Mi 
and Ci in ajiQ{H] M, C) with arrowhead pointing to j on the path. By what we showed 
before in the proof of Theorem 1 in H there is an m-connccting path between i and j 
given M U Mi and C U Ci with arrowhead pointing to j on the path. 

Lemma 1 implies that in aRciH; M U Mi, CUCi) there is an ij-edge with arrowhead 
pointing to j. We showed before that j an(C) in H. In addition, in H, j ^ an(Ci): 
Suppose, for contradiction, that j S an(Ci) in H. This direction-preserving path is also 
direction-preserving in aRciH; M, C) unless possibly a node t on the path is in M or in 
C. In the former case one can skip t and obtain a direction-preserving path. In the latter 
case j G an(C) in H, which is not permissible. Therefore, in ajidH] M, C), j G an(Ci). 
Hence, since j ^ an(C) in H, j £ an(Ci) in asciH; M, C), a contradiction. 

Therefore, j ^ an(C U Ci) in H, and the ij-edgc has arrowhead pointing to j in 
asGiH;MUMi,CUCi). □ 

Proof of Theorem 4- By what we proved in Theorem 2 it is enough to show between 
i and j there is an m-connecting path given Ci in a rq (H; M, C) if and only if there is 
an m-connecting path given Ci in asaiH; M, C). 

(=5>) Consider an m-connecting path given Ci in ajidH; M, C) between i and j. There 
obviously exists an m-connecting path tt given Ci in anoiH ; M,C). Now by step 2 of 
the algorithm non-collider nodes remain non-collider. In addition, if a collider V {h, fc, I) 
on TT turns into non-collider then k G an(C) and therefore by step 1 of the algorithm 
there is an endpoint-identical hl-edge generated that can be used instead of the V to 
generate an m-connecting path. Moreover, if the collider node k is in an(Ci), and on 
the direction-preserving path an arrow turns into a line then k G an(C) in H and once 
again there is an hl-edge to be used instead of the V to establish an m-connecting path. 
Therefore, there is an m-connecting path between i and j given Ci in asaiH; M, C). 

Consider an m-connecting path tt given Ci in asciH] M,C) between i and 
j. By step 2 of the algorithm, collider nodes on tt in asciH; M,C) are colliders in 
audH; M, C). In addition, if a non-collider V {h, k, I) on tt is collider in anciH; M, C) 
then k G an(C) in H and therefore by step 1 of the algorithm there is an endpoint- 
identical hl-edge that can be used instead of the V to generate an m-connecting path in 
afic{H; M, C). Moreover, if the collider node fc on tt is in an(Ci), and on the direction- 
preserving path an arrow is an arc in aRG{H; M,C) then k G an(C) in H and once 
again there is an hl-edge to be used instead of the V to establish an m-connecting path. 
Therefore, there is an ?7i-connecting path between i and j given Ci in anciH; M, C). □ 

Proof of Proposition 6. To prove that the graph generated by Algorithm 3 is an 
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AG, first notice that graphs generated by step 1 of Algorithm 3 are summary graphs. 
Therefore, since steps 2 and 3 do not generate any hnes, it is enough to prove that steps 
2 and 3 of the algorithm remove all subgraphs where there is an arc with one endpoint 
that is an ancestor of the other endpoint in the generated summary graph, and that these 
do not generate any direction-preserving cycles. 

Step 3 of the algorithm removes all such subgraphs. Step 2 does not generate any 
direction-preserving cycles by adding an arrow to the graph: Consider the first iteration 
of the algorithm, where, for contradiction, a direction-preserving cycle is generated. If it 
is generated by generating an arrow from i to j then we know that there is i — < > j, 
where k S an(j). Denote the direction-preserving path from k to j by tti and the direction- 
preserving path from j to i which, together with the generated ij-arrow, establishes a 
direction-preserving cycle by 772- It is seen that in the the previous iteration of the 
algorithm (7r2, fc,7ri) is a direction-preserving cycle, a contradiction. 

Step 3 does not generate any direction-preserving cycles by replacing an arc by an 
arrow: Consider the first iteration of the algorithm, where, for contradiction, a direction- 
preserving cycle is generated. If it is generated by replacing an ij-arc by an arrow from i to 
j then we know there is a direction-preserving path tti from i to j. Denote the direction- 
preserving path from j to i which, together with the generated zj-arrow, establishes 
a direction-preserving cycle by 7r2. It is seen that in the the previous iteration of the 
algorithm (7ri,7r2) is a direction-preserving cycle, a contradiction. □ 

We use the following lemma to prove Theorem 5. For more descriptive proofs for the 
following results, see [11]. 

Lemma 5. Let H be a summary graph and M and C he two subsets of its node set. It 
holds that aAGiasG.AG{H); M, C) = aAoiH; M, C). 

Proof. There are two differences between H and asG.AciH): (1) For an ij-aic such 
that j g an(z) in H, there is an arrow from j to i replaced in ascAoiH)] (2) for a 
primitive inducing path tt between i and j in iJ, there is an endpoint-identical ij-edge 
in asG.AG{H). 

Notice that an{C) is the same in both graphs. After applying step 1 of Algorithm 2 
(a part of step 1 of Algorithm 3) , for each difference, the following occurs: 

(1) This step of the algorithm may generate further differences for (1) if there is a 
fcj-edge with an arrowhead pointing to j and j S CUan(C). In this case there is a fci-edge 
in H. However, such an edge already exists in asG.AG{H) since {k,j,i) in H generates 
an edge by ascAc; 

(2) This step of the algorithm may generate further differences for (2) if tt has more 
than three nodes and one of the two following cases occurs: (a) there is an arrowhead 
pointing to j on tt, there is a fcj-edge with an arrowhead pointing to j, and j € CUan(C); 
(b) there is a kj-edge with no arrowhead pointing to j, and j S M. In both cases, by 
using the fc)-V a ki-edge is generated in ascAciH). In H, by using the {h,j, A;)-V, 
where h is the node adjacent to j on tt, a kh-edge is generated, which establishes an 
endpoint-identical primitive inducing path between i and k in asG.AG^H)] 
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After applying step 2 of Algorithm 2 the following occurs: (1) This step of the algorithm 
may generate further differences for (1) if j S C U an(C). In this case ij-arc in H turns 
into an arrow from j to i, which, however, already exists in asG.AoiH); (2) This step of 
the algorithm may generate further differences for (2) if any of the nodes on tt, say is in 
C U an(C). However, in H, by the previous step of the algorithm, the nodes adjacent to 
/ on TT have become adjacent, and established a shorter primitive inducing path between 
i and k (or j). 

Hence, thus far, the differences between the two generated graphs are the same as the 
differences between H and olsg.ag{.H). Therefore, by applying steps 2 and 3 of Algorithm 
3, the same graphs will be generated. □ 

Proof of Theorem 5. Using Theorem 3, Proposition 7, and Lemma 5, we have the 
following: 

aAG{aAG{H; M, C); Mi, Ci) aAG{asG.AG{asG{H; M, C)); Mi, Ci) = 

aAG(asG(i?;M,C);Mi,Ci) =asG.AG(asG(asG(i?;M,C);Mi,Ci)) = 
asG.AGiasGiH]M U Mi,C UCi)) = aAGiH] M U Mi,C U Ci). 

□ 

Proof of Theorem 6. By what wc proved in Theorem 4 it is enough to show between 
A and B there is an m-connecting path given Ci in asG{H; M, C) if and only if there is 
an TO-connecting path given Ci in aAG^H; M, C). 

Let {asG{H; M,C) = Ho,Hi,. . . ,Hm) be intermediate graphs that have each been 
generated by adding one edge to the previous graph by step 2 of Algorithm 3. In addition, 
let {Hm = Hq, H[, . . . , = aAG{H; M, C)) be intermediate graphs that have each been 
generated by replacing one edge in the previous graph by step 3 of Algorithm 3. 

Suppose that in the step between Hp and i?p+i an arrow from j to i or an arc between 
i and j for V j — ^fc < > i or V j < > k < > i, when k G an(i), is generated. It holds that 
there is an m-connecting path between between A and B given Ci in Hp if and only 
if there is an m-connecting path between between A and B given Ci in -ffp+i. This is 
because if fc 6 Ci or one of the descendants of k on the direction-preserving path from 
fc to i is in Ci then the ij-edge and the V j < > k < > i can be interchanged on the 
m-connecting path. If these nodes are not in Ci then the ij-edge and the path made 
up by the jfc-edge and the direction-preserving path from k to i can be interchanged. 
By induction in one direction and reverse induction in the other direction we conclude 
that there is an m-connecting path given Ci in asG{H\ M, C) if and only if there is an 
m-connecting path given Ci in Hm- 

Now suppose that in the step between H'^, and H^,^^ an arrow from j to i has been 
replaced by an arc between j and i, where j S an(i). The only interesting case here is 
when there is an edge between j and another node I with arrowhead pointing to j. In 
this case an edge has been already generated between I and i by step 2 of the algorithm. 
Therefore, again by induction in one direction and reverse induction in the other direction 
the result follows. □ 
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